Papers with bias detection
Towards Detecting Political Bias in Hindi News Articles (2022.acl-srw)
Copied to clipboard
| Challenge: | Political propaganda in recent times has been amplified by media news portals through biased reporting, creating untruthful narratives on serious issues . a dataset for this task was not available, therefore we developed a transformer-based transfer learning method to fine-tune the pre-trained network on our data. |
| Approach: | They propose a transformer-based transfer learning method to fine-tune the pre-trained network on the data for this bias detection. |
| Outcome: | The proposed method fine-tunes the pre-trained network on the data to detect political bias in Hindi news articles. |
LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation (2026.eacl-demo)
Copied to clipboard
| Challenge: | Existing work on bias evaluation includes benchmark datasets and automated detection methods. |
| Approach: | They propose an open-source web application for side-by-side comparison of LLM outputs with real-time bias analysis. |
| Outcome: | The open-source application compares LLM outputs with real-time bias analysis. |
Content-Based Conflict of Interest Detection on Wikipedia (L18-1)
Copied to clipboard
| Challenge: | Conflict-of-Interest (CoI) editing is a problem on Wikipedia that is highly subjective . a key feature of Wiki sites is to allow people from all over the world to add or modify articles anonymously and without consequence. |
| Approach: | They frame CoI detection as a binary classification problem and explore features for it . they find that stylometric features outperform other types of features and give an F-measure of 0.63 . |
| Outcome: | The proposed method outperforms other features and gives an F-measure of 0.63 . the proposed method is not certain that the set of non-CoI articles contains any CoI articles . |
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023.findings-acl)
Copied to clipboard
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, Jimmy Huang
| Challenge: | Currently, the evaluation of large language models (LLMs) such as ChatGPT in academic datasets is difficult due to the difficulty of evaluating the generative outputs produced by this model against the ground truth. |
| Approach: | They evaluate ChatGPT across 140 tasks and analyze 255K responses it generates in academic datasets. |
| Outcome: | The proposed model performs well on 140 tasks and generates 255K responses in these datasets. |
A Study of Nationality Bias in Names and Perplexity using Off-the-Shelf Affect-related Tweet Classifiers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research shows that named entities influence PLMs in many applications. |
| Approach: | They propose a method to quantify biases associated with named entities from various countries using Twitter data instead of templates or specific datasets. |
| Outcome: | The proposed method shows positive biases related to the language spoken in a country across all classifiers. |
IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing studies on social media bias detection focus on fine-tuning models specific to particular datasets and testing them on corresponding test sets. |
| Approach: | They propose a general bias detection framework, IndiVec, built upon large language models and vector databases. |
| Outcome: | The proposed framework outperforms baseline methods on four political bias datasets and provides explicit top-k indicators to interpret bias predictions. |
On the Mutual Influence of Gender and Occupation in LLM Representations (2025.acl-long)
Copied to clipboard
| Challenge: | We examine LLM representations of gender for first names in various occupational contexts to study how occupations and the gender perception of first names influence each other mutually. |
| Approach: | They examine LLM representations of gender for first names in various occupational contexts and examine how occupations and the gender perception of first names influence each other mutually. |
| Outcome: | The representations shift with the occupational context and are influenced by stereotypically feminine or masculine occupations. |
From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to evaluate code generation bias focus on overt discrimination through simple conditional statements. |
| Approach: | They examine ML pipelines that exhibit substantially greater bias than simple conditionals . they challenge simple conditional statements as valid proxies for bias evaluation . |
| Outcome: | The proposed model underestimates real-world bias in generating machine learning pipelines . the model maintains equal performance on simple conditionals and ML pipelines, the study shows . |
Attribution-Guided Multi-Object Hallucination and Bias Detection in Vision-Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods struggle with multi-object grounding because language priors dominate visual evidence, causing hallucinated or biased objects to produce attention distributions or similarity scores nearly indistinguishable from those of real objects. |
| Approach: | They propose a Shapley value-based attribution framework that uses Kernel SHAP and multi-layer fusion to detect hallucinated and biased objects. |
| Outcome: | Evaluated on ADE and COCO datasets, SHAPLENS improves hallucination detection accuracy by 8–12% and F1 by 10–14% over baselines. |
Using Social and Linguistic Information to Adapt Pretrained Representations for Political Perspective Identification (2021.findings-acl)
Copied to clipboard
| Challenge: | a new framework for political perspective detection is proposed to improve text training costs . current deep learning models lack the ability to focus on text span for bias detection . |
| Approach: | They propose a framework that pretrains the text model using social and linguistic contexts . they demonstrate that the framework improves performance by identifying bias-related text spans based on entity mentions and news sharing . |
| Outcome: | The proposed framework improves on two news bias datasets and improves performance on the general source and task. |
Trade-Offs Between Fairness and Privacy in Language Modeling (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing research suggests that privacy preservation comes at the price of worsening biases in classification tasks. |
| Approach: | They propose to incorporate privacy preservation and de-biasing techniques into training text generation models to investigate the trade-off between the two dimensions. |
| Outcome: | The proposed model improves on bias detection, privacy attacks, language modeling, and performance on downstream tasks. |
Investigating Bias in LLM-Based Bias Detection: Disparities between LLMs and Human Perception (2025.coling-main)
Copied to clipboard
| Challenge: | Detecting media bias is critical due to the spread of misinformation and disinformation on social media platforms. |
| Approach: | They investigate the presence and nature of bias within large language models and its consequential impact on media bias detection. |
| Outcome: | The proposed debiasing strategies include prompt engineering and model fine-tuning. |
Stereotype Detection as a Catalyst for Enhanced Bias Detection: A Multi-Task Learning Approach (2025.findings-acl)
Copied to clipboard
| Challenge: | a new study addresses bias and stereotypes in language models by exploring how learning them together improves performance. |
| Approach: | They propose a dataset for bias and stereotype detection that integrates religion, gender, socio-economic status, race, profession, and others. |
| Outcome: | The proposed dataset compares encoder-only models and fine-tuned decoder- only models . the results show that learning stereotypes together improves bias detection . |
LLMs can be easily Confused by Instructional Distractions (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) show exceptional skill in instruction following tasks, but can become vulnerable when they are required to disregard instructions. |
| Approach: | They propose a benchmark to assess LLMs' performance under instructional distraction. |
| Outcome: | The proposed benchmark categorizes real-world instances of instructional distraction and evaluates LLMs across four instruction tasks: proofreading, rewriting, translation, and style transfer—alongside five input tasks: reasoning, code generation, mathematical reasoning, bias detection, and question answering. |
Large Language Models as Reader for Bias Detection (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Traditional methods analyze text from the writer’s perspective, leaving the reader’s viewpoint underexplored. |
| Approach: | They investigate whether large language models can be leveraged as readers for bias detection by generating reader-perspective comments. |
| Outcome: | The proposed model performs comparable to GPT4's in detecting bias in media content. |
Probing Political Ideology in Large Language Models: How Latent Political Representations Generalize Across Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models encode rich internal representations of political ideology, but it remains unclear how these representations contribute to model decision-making. |
| Approach: | They apply inference-time interventions to steer a decoder-only transformer along learned ideological directions . they find that learned ideological representations generalize well to bias detection, but not as well to voting simulations . |
| Outcome: | The proposed model steers a transformer along learned ideological directions . political bias detection, voting preference simulation and bias neutralization are tested . |